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>A  second  project  involved  the  development  of  a  computerized  technique  for  generating 
two-dimensionai  maps  of  cerebral  cortex.  An  algorithm  based  on  simulated  annealing 
procedures  was  used  to  construct  a  complete  map  of  primary  visual  cortex,  thereby 
demonstrating  its  suitability  for  dealing  with  anatomical  data  from  highly  convoluted 
regions  of  cortex. 

A  third  project  involved  the  use  of  voltage-sensitive  dyes  to  monitor  activity  patterns 
in  visual  cortex.  This  technique  offers  great  promise  for  analyzing  the  organization  of 
large  neurad  ensembles  with  high  spatial  and  temporal  resolution.  l[^exA^^^o^dCo 
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The  research  supported  by  this  contract  has  been  aimed  at  elucidating  the 
functional  organization  of  mammalian  visual  cortex.  We  use  a  combination  of 
physiological  and  anatomical  techniques  to  identify  different  visual  areas  and  to 
characterize  how  information  processed  within  each  area  and  distributed  among  areas. 
There  have  been  three  major  components  to  our  overall  effort  during  the  past  year.  The 
first  project  is  an  analysis  of  texture  vision  as  assayed  by  monitoring  the  activity  of 
single  neurons  in  areas  VI  and  V2  of  the  macaque  in  response  to  texture  patterns  of  the 
type  developed  by  Julesz  and  collaborators.  The  second  project  is  the  development  of  a 
computerized  technique  to  generate  two-dimensional  maps  of  cerebral  cortex,  for  use  in 
determining  the  functional  organization  of  different  visual  areas.  The  third  project  is 
the  application  of  an  optical  technique  for  monitoring  the  physiological  activity  of  visual 
cortex  with  high  spatial  and  temporal  resolution.  Substantial  progress  has  been  made  on 
each  of  these  projects,  as  described  in  the  sections  below. 

Physiology  of  Texture  Vision 

Our  sense  of  vision  involves  simultaneous  operation  of  two  distinct  components;  an 
attentive  system  that  provides  detailed  scrutiny  of  one  object  at  a  time  in  the  visual 
field,  and  a  pre-attentive  system  that  continuously  scans  the  entire  visual  field  in 
parallel  and  selects  items  of  particular  salience  for  further  scrutiny  by  the  attentive 
system  (Julesz,  1984).  The  pre-attentive  system  appears  to  specialize  in  providing 
information  about  visual  texture,  i.e.,  the  arrangement  of  features  that  provide  surfaces 
with  their  characteristic  appearance.  One  of  the  keys  to  Julesz'  recent  approach  has 
been  to  simpify  the  analysis  of  texture  vision  by  concentrating  on  patterns  that  consist  of 
an  array  of  discrete,  overlapping  elements,  called  textons.  In  essence,  this  approach  has 
allowed  a  detailed  dissection  of  which  characteristics  of  a  pattern  are  important  in 
texture  discrimination  and  which  characteristics  are  irrelevant. 


Our  general  aim  has  been  to  identify  and  characterize  cells  in  monkey  visual  cortex 
that  may  be  involved  in  mediating  the  perception  of  texture.  We  have  used  the  texture 


patterns  devised  by  Julesz  and  have  used  stimulation  paradigms  that  are  based  on  the 
concepts  that  have  emerged  from  his  psychophysical  studies.  In  the  preliminary 
experiments  carried  out  to  date,  we  concentrated  on  two  paradigms.  The  first  was 
directed  at  attention-related  mechanisms  for  detecting  the  presence  of  local  texture 
differences  (texture  gradients).  The  second  was  directed  at  the  analysis  of  "texture 
contours"  whose  presence  is  signaled  by  borders  across  which  there  is  a  texture 
gradient.  This  project  has  been  carried  out  by  Ted  DeYoe,  Jim  Knierim,  Dov  Sagi,  Bela 
Julesz,  and  myself. 

Our  experimental  preparation  is  the  anesthetized,  paralyzed  macaque  monkey.  We 
made  extracellular  recordings  from  single  nerve  cells  in  areas  VI  and  V2,  using  recording 
procedures  standard  for  this  laboratory  (Manusell  and  Van  Essen,  1983).  Visual  stimuli 
were  generated  on  a  Hewlett-Packard  CRT  monitor  using  a  software  and  hardware 
system  provided  on  loan  by  Bela  Julesz.  Data  acquisition  and  analysis  was  carried  out  on 
our  existing  PDP  11/34  computer. 


Texture  Gradients  using  Static  Center-Surround  Patterns 

The  salience  of  any  given  texton  to  the  pre-attentive  system  depends  critically  on 
the  context  in  which  it  is  presented,  i.e.,  its  relationship  to  nearby  textons.  A  texton  of 
any  sort  is  immediately  obvious  if  it  is  presented  in  complete  isolation  or  if  it  is 
presented  within  a  field  of  textons  having  different  attributes,  such  as  color  or 
orientation.  If,  however,  it  is  embedded  within  a  field  of  identical  textons,  it  does  not 
stand  out. 

We  tested  for  neural  correlates  of  these  psychophysical  observations  using  patterns 
consisting  of  a  central  region  lying,  within  the  receptive  field  of  the  neuron  under  study, 
and  a  larger  surround  region  lying  completely  outside  the  neuron's  "classical”  receptive 
field.  Textons  consisted  of  small  elongated  bars  of  light,  and  each  texton  was  placed 
either  at  the  cell's  preferred  orientation  or  perpendicular  to  it.  All  possible  combinations 
of  center  and  surround  patterns  were  tested  on  each  cell,  and  results  from  one 
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particularly  interesting  cell  are  illustrated  in  Figure  1.  The  bottom  row  shows  the  eight 
different  stimulus  patterns;  the  middle  row  shows  peristimulus  response  histograms 
determined  from  10  pseudorandomly  interleaved  repetitions  of  each  stimulus;  and  the  top 
row  shows  bar  histograms  of  the  average  response  rate.  The  cell  responded  well  to 
stimulus  C,  a  single  bar  of  the  optimal  orientation  and  placed  within  the  receptive  field. 
It  also  responded  well  to  stimulus  A,  consisting  of  the  same  central  bar  surrounded  by 
bars  of  the  orthogonal  orientation.  In  contrast,  there  was  only  a  weak  response  to 
stimulus  B,  in  which  the  central  stimulus  was  surrounded  by  bars  of  identical 
orientation.  The  remaining  portion  of  the  figure  illustrates  several  control  patterns, 
demonstrating  that  the  cell  did  not  fire,  for  example,  to  the  surround  when  presented 
without  the  central  bar.  In  short,  the  response  of  this  cell  mimic  several  key  features  of 
human  texture  vision:  the  response  is  large  if  the  stimulus  within  the  receptive  field  is 
salient  to  the  pre-attentive  system,  either  because  of  a  texture  gradient  (stimulus  A)  or  a 
combined  texture  and  luminance  gradient  (stimulus  C),  whereas  the  response  is  small  if 
the  central  bar  is  not  pre-attentiveJy  salient,  because  there  is  no  texture  gradient 
(stimulus  B). 

Thus  far  we  have  obtained  good  recording  data  from  37  cells  in  VI  and  V2  when 
using  this  paradigm.  A  pronounced  suppressive  effect  of  the  stimulus  surrounded  was 
found  for  at  least  one  stimulus  configuration  in  16  of  22  V2  cells  and  8  of  15  Vi  cells. 
For  some  cells  the  surround  effect  was  non-specific.  More  interestingly,  for  7  cells  in  V2 
and  2  cells  in  V 1  the  interaction  was  dependent  on  both  center  and  surround:  there  was 
substantial  suppression  for  matching  orientations  but  little  or  none  for  contrasting 
orientations.  Thus,  the  responses  of  such  cells  correlate  with  the  perceptual  salience  of 
the  center  texture  element. 

Responses  to  Moving  Texture  Contours 

A  texture  pattern  that  moves  relative  to  its  background  suddenly  stands  out,  a  fact 
that  works  to  the  disadvantage  of  prey  and  to  the  advantage  of  predators.  We  have 


analyzed  the  responses  of  cell  in  VI  and  V2  to  moving  texture  patterns  of  the  type 
illustrated  in  the  upper  part  of  Figure  2.  A  "texture  bar,"  consisting  of  a  vertical  column 
of  short  horizontally  oriented  textons,  is  embedded  in  a  field  of  vertically  oriented 
textons  (Frame  1).  On  each  successive  frame,  the  texture  bar  moves  one  column  to  the 
right.  To  a  human  observer,  this  sequence  appears  as  a  bar  moving  steadily  to  the  right 
("apparent  motion"),  with  a  smoothness  that  depends  on  the  spatial  separation  of  columns 
and  the  temporal  separation  of  frames. 

The  lower  portion  of  Figure  2  illustrates  the  responses  of  a  cell  in  area  V2  to  the 
apparent  motion  of  a  texture  bar.  The  cell  responded  best  to  rightward  motion  of  a 
texture  bar  consisting  of  vertical  textons,  but  there  was  also  a  good  response  when  the 
orientation  of  the  textons  in  the  texture  bar  and  surround  were  reversed.  However,  there 
was  amuch  weaker  response  to  downward  motion  of  a  horizontal  texture  bar,  irrespective 
of  texton  orientation.  Thus,  the  responses  of  this  cell  were  determined  mainly  be  the 
configuration  of  the  texture  bar,  rather  than  the  composition  of  the  individual  texture 
elements. 

In  more  recent  experiments,  we  have  obtained  similar  results  using  patterns 
consisting  of  textons  at  oblique  orientations  to  the  texture  bar,  thereby  eliminating  any 
subtle  luminance  cues  that  might  otherwise  be  used  for  discriminating  apparent  motion. 
These  and  other  controls  have  convinced  us  that  the  responses  of  many  cells  in  V2  (26  of 
40  tested)  encode  useful  information  about  moving  texture  patterns.  In  contrast,  only  2 
of  16  cells  in  VI  that  were  tested  in  a  comparable  fashion  responded  to  the  motion  of  the 
texture  bar  rather  than  the  texture  elements.  Much  more  experimentation  will  be 
necessary,  however,  to  fully  clarify  the  nature  of  this  information  encoding  in  V2  and  V2 
to  establish  its  relevance  to  texture  perception. 

Computerized  Mapping 


The  convolutions  of  cerebral  cortex  in  many  species  imposes  major  difficulties  in 
carrying  out  precise  analyses  of  the  functional  organization  of  different  visual  areas. 
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For  about  a  decade  we  have  been  able  to  circumvent  many  of  these  difficulties  by 
displaying  our  experimental  findings  on  two-dimensional  maps  of  the  cortical  surface 
(Van  Essen  and  Zeki,  197S;  Van  Essen  and  Maunsell,  1980).  Our  original  procedure  for 
generating  these  maps  was  an  entirely  manual  method,  and  it  had  the  disadvantages  of 
being  difficult  to  learn,  tedious  to  implement,  and  limited  in  overall  accuracy.  It  has 
long  been  one  of  our  goals  to  develop  a  completely  computerized  procedure  for 
constructing  two-dimensional  maps,  and  George  Carman  in  my  lab  has  now  succeeded  in 
this  endeavor. 

The  initial  database  for  the  computerized  mapping  procedure  comes  from  the 
contours  of  cortical  layer  4  (the  middle  layer)  from  histological  sections  taken  at  closely 
spaced  intervals  (0.5  mm)  through  the  hemisphere.  These  contours  are  digitized  on  a 
graphics  tablet  and  stored  in  disk  memory  on  our  Masscomp  535  computer.  Each  of  these 
contours  is  smoothed  and  sampled  at  intervals  of  0.5  mm.  The  resultant  set  of  points 
form  a  three-dimensional  reference  surface  used  in  the  mapping  procedure.  (Each  point 
has  an  x,y  coordinate  from  its  location  within  the  section  and  a  z  coordinate  from  the 
level  of  the  particular  section  to  which  it  belongs.)  For  each  point  on  this  reference 
surface,  a  corresponding  point  is  created  on  a  two-dimensional  mapping  surface.  The 
initial  configuration  of  the  two-dimensional  surface  is  not  crucial;  the  algorithm 
succeeds  when  all  points  on  the  lattice  are  started  at  a  singularity,  so  that  topologically 
correct  ordering  of  points  can  be  restored  even  when  initially  absent  from  the  map. 

The  heart  of  the  mapping  algorithm  is  an  iterative  procedure  that  gradually 
transforms  the  2-D  map  into  its  proper  configuration.  This  is  achieved  using  a  metric  of 
local  distortion,  called  the  energy,  which  reflects  the  deviation  of  the  local  geometry  on 
the  2-D  map  from  the  desired  geometry  present  in  the  3-D  surface.  The  basic  strategy  is 
to  randomly  move  the  points  on  the  map  so  as  to  minimize  the  deviations  of  lengths  and 
angles  on  the  2-D  map  from  the  corresponding  distances  and  angles  on  the  3-D  reference 
surface.  This  comparison  between  actual  and  desired  local  relationship  for  one  lattice 
point  and  its  four  neighbors  is  illustrated  in  Figure  3.  The  sum  of  energies  for  all  points 
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yields  a  global  energy,  which  must  be  minimized  in  order  to  generate  an  optimal  2-D  map 
of  the  reference  surface. 

To  find  this  global  energy  minimum,  we  subject  the  Z-D  map  to  stimulated 
annealing  (Kirkpatrick  et  al.,  1983).  In  our  implementation  of  this  iterative  algorithm,  a 
point  is  chosen  randomly  from  the  lattice,  and  a  small  random  displacement  of  that  point 
is  attempted  within  the  plane  of  the  map.  Displacements  resulting  in  a  more  accurate 
local  geometry  (decrease  in  energy)  are  always  accepted,  while  those  resulting  in  a  less 
accurate  local  geometry  (increase  in  energy)  are  accepted  with  a  probability  dependent 
upon  a  control  parameter  termed  the  "temperature.”  Initially,  the  temperature  is  set 
high  in  order  to  allow  positions  of  points  of  the  map  to  become  randomized.  The 
temperature  is  then  gradually  reduced  over  successive  iterations,  allowing  the  points  to 
approach  their  optimal  local  geometries  and  resulting  in  the  emergence  of  an  optimal 
global  conformation  of  the  map. 

We  initially  developed  and  tested  this  algorithm  .by  generated  2-D  maps  of 
mathematically  precise  3-D  geometrical  surfaces  such  as  cones  and  hemispheres.  For 
example.  Fig.  4  shows  three  stages  in  the  generation  of  an  unfolded  map  of  a 
hemisphere.  The  lattice  on  the  top  left  is  at  an  early  stage  of  annealing,  and  parts  of  the 
map  are  not  even  in  a  topologically  correct  order.  The  lattice  on  the  top  right  is 
topologically  correct,  but  still  has  considereible  local  disorder.  Finally,  the  lattice  on  the 
bottom  represents  a  map  essentially  at  its  global  energy  minimum,  achieved  after  ca.  258 
iterations  and  1.5  hours  of  computational  time. 

The  current  mapping  algorithm  appears  to  be  fairly  robust,  in  that  it  can  generate 
accurate  2-D  maps  from  a  variety  of  3-D  surfaces,  and  it  is  flexible  with  regard  to  the 
starting  configuration  of  the  lattice  and  the  precise  sequence  of  temperature  changes 
during  annealing.  Reaching  this  level  of  performance  was  not  an  easy  feat,  however. 
There  are  many  subtleties  to  the  algorithm  that  must  be  recognized  and  respected,  and 
experience  has  revealed  that  even  minor  flaws  can  lead  to  highly  aberrant  maps  as  the 
final  product.  Thus,  the  testing  of  the  algorithm  on  mathematically  well-defined 
surfaces  was  crucial  to  its  eventual  success. 


We  next  applied  this  approach  to  real  neuroanatom ical  data,  starting  with  a  map  of 
striate  cortex  (VI).  We  used  contours  from  the  same  hemisphere  that  had  previously 
been  mapped  by  LeVay  et  al.  (1985)  using  our  standard  manual  procedure.  Figure  5  shows 
an  intermediate  stage  (left)  in  the  annealing  of  this  map,  and  the  final,  minimum-energy 
stage  (right),  reached  after  250  iterations  and  24  hours  of  computation.  In  this  display 
the  continuous  lines  represent  the  contours  associated  with  individual  sections  through 
the  cortex;  the  connections  between  lattice  points  on  adjacent  sections  are  omitted  for 
clarity,  even  though  they  were  equally  important  in  generating  the  map. 

The  computerized  2-D  map  is  similar  to  our  earlier  manual  version  in  its  general 
configuration,  but  it  has  several  important  advantages.  Both  have  some  residual 
distortions  that  cannot  be  removed  because  of  the  intrinsic  curvature  of  the  cortex  (Van 
Essen  and  Maunsell,  1980).  However,  the  distortions  on  the  computerized  map  are 
smaller;  moreover,  their  magnitude  can  be  calculated  and  displayed  precisely.  The 
computerized  procedure  is  more  reliable,  does  not  depend  on  subjective  judgements,  and 
frees  the  neuroanatomist  from  having  to  learn  a  tricky  and  tedious  procedure. 

There  remains  a  considerable  amount  of  work  left  before  all  stages  of 
computerized  map-making  are  refined  and  completely  automated.  Of  particular 
importance  is  the  procedure  for  linking  corresponding  points  on  adjacent  sections,  in 
order  to  create  a  completely  connected  lattic  for  the  3-D  reference  surface.  We  are 
currently  exploring  several  possible  options  for  most  efficiently  and  reliably  executing 
this  task.  Ultimately,  we  plan  to  have  a  complete  package  of  algorithms  that  will  be 
made  available  to  interested  users  in  the  neuroscience  community. 

Optical  Monitoring  of  Brain  Activity 

Recently,  a  promising  new  technique  has  been  developed  lor  monitoring  brain 
activity  using  an  optical  technique  rather  than  standard  electrical  recording  methods. 
This  approach  relies  on  the  use  of  special  organic  dyes  which,  when  applied  to  the  brain, 
change  their  absorbance  or  fluorescence  in  proportion  to  the  degree  of  local  neural 


activity  (Orbach  and  Cohen,  1983;  Grinvald,  1985).  By  mounting  an  array  of 
photodetectors  onto  a  microscope  viewing  the  brain  surface,  it  is  possible  to  monitor  the 
level  of  neural  activity  simultaneously  at  each  of  100  different  locations.  This  technique 
offers  a  combination  of  high  temporal  and  spatial  resolution  that  is  unavailable  with  any 
other  recording  technique. 

During  the  past  year.  Dr.  Harry  Orbach  has  been  involved  in  setting  up  the 
experimental  apparatus  for  this  project  and  carrying  out  pilot  studies.  This  has  entailed 
extensive  design  and  assembly  of  the  various  hardware  and  software  components  that  are 
needed  for  the  specific  projects  we  will  be  undertaking,  with  the  hardware  being 
constructed  by  engineer  Herb  Adams  and  the  software  developed  by  Dave  Bilitch.  The 
major  system  components  include:  1)  a  rat  head-holding  device  which  keeps  the  animal 
securely  and  stably  positioned,  and  which  is  compact  enough  to  fit  on  a  microscope  stage; 
2)  a  transparent  chamber  for  viewing  the  exposed  cortical  surface  while  minimzing 
cortical  pulsations;  3)  a  microdrive  for  advancing  electrodes  into  cortex  to  allow 
simultaneous  optical  and  electrical  measurements;  4)  modifications  of  a  Zeiss  University 
microscope  to  allow  insertion  of  the  animal  under  the  objective  and  positioning  of  the 
photodector  array  on  a  vertical  phototube;  5)  linkage  of  100  photodetectors  through 
individual  low-noise  amplifiers  to  the  multiplex  A/D  port  of  the  Masscomp;  6) 
development  of  software  for  100-channel  data  acquisition,  analysis  and  graphical  display; 
7)  construction  of  a  projection  system  for  presenting  simple  patterns  (bars  and  edges  of 
light)  for  use  as  visual  stimuli;  and  8)  programming  of  an  IRIS  video  graphics  system  for 
presentation  of  more  comlex  stimulus  patterns  (e.g.,  oriented  gratings). 

These  various  developmental  stages  are  now  largely  completed,  and  we  have  been 
able  to  carry  out  preliminary  experiments.  Most  experiments  to  date  have  been  done  on 
rat  visual  cortex.  One  major  objective  has  been  to  obtain  simultaneous  optical  and 
electrical  recordings  in  response  to  restricted  visual  stimuli.  We  feel  that  such  an 
analysis  will  provide  crucial  but  currently  unavailable  information  concerning  the 
relationship  between  electrically  and  optically  mediated  events  in  the  cortex.  To  date, 


we  have  gotten  good  electircal  responses  in  some  experiments  and  clear  optical  signals  in 
other  experiments;  for  a  variety  of  technical  reasons  the  combined  experiment  has  not 
yet  worked,  but  we  expiect  it  to  succeed  soon. 

Another  project  already  underway  is  to  search  for  new  dyes  that  give  large  optical 
signals,  as  the  low  signal-to-noise  is  a  crucial  limiting  factor  in  most  CNS  experiments. 
We  are  planning  to  try  a  variety  of  dyes,  some  fast-acting  for  good  temporal  resolution, 
but  also  some  that  integrate  over  longer  time  periods,  thereby  trading  temporal 
resolution  for  the  prospect  of  a  stronger  signal. 
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